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Unmanned Aircraft System (UAS) Delegation of Separation 

in NextGen Airspace 

Caitlin Kenny 1 , Robert J. Shively 2 , Kevin Jordan 3 

ABSTRACT 

The purpose of this study was to determine the feasibility of unmanned aircraft 
systems (UAS) performing delegated separation in the national airspace system 
(NAS). Delegated separation is the transfer of responsibility for maintaining 
separation between aircraft or vehicles from air navigation sendee providers to 
the relevant pilot or flight operator. The effects of delegated separation and 
traffic display information level were collected through performance , workload, 
and situation awareness measures. 

The results of this study show benefits related to the use of conflict detection 
alerts being shown on the UAS operator’s cockpit situation display (CSD), and to 
the use of full delegation. Overall, changing the level of separation responsibility 
and adding conflict detection alerts on the CSD was not found to have an adverse 
effect on performance as shown by the low amounts of losses of separation . The 
use of conflict detection alerts on the CSD and full delegation responsibilities 
given to the UAS operator were found to create significantly reduced workload, 
significantly increased situation awareness and significantly easier 
communications between the UAS operator and air traffic controller without 
significantly increasing the amount of losses of separation. 


1.0 Introduction 

A simple definition of unmanned aircraft systems (UAS) is to describe them as essentially an aircraft 
with the flight crew and support crew removed from the vehicle and placed at a ground control 
system with a connecting computer system and radio link used for command and control purposes 
(Austin, 2010). The demand for public access of UAS in the U.S. National Airspace (NAS) has 
grown dramatically. Since 2004, the number of public requests to fly UAS has increased over 900% 
(JPDO, 2012b). Projections of development from 2010 to 2019 predict over 20,000 UAS created in 
the United States and 35,000+ created worldwide (Teal Group, 2009). The Federal Aviation 
Administration (FAA) Modernization and Reform Act of 2012 addresses this increasing demand by 
requiring full integration of UAS into the NAS by 2015 (FAA, 2012a). Many concerns exist as to 
the best way to integrate UAS into the NextGen environment within the FAA mandated time frame. 


1 Hewlett Packard. 

2 NASA Ames Research Center, Moffett Field, California. 

3 San Jose State University Foundation, San Jose, California. 
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1.1 The Next Generation Air Transportation System 

The Next Generation Air Transportation System (NextGen) is an overhaul of the NAS that is aimed 
to create a safer, more convenient and dependable airspace system using satellite based information 
by 2025 (FAA, 2012b). The use of satellite information will allow for shorter routes, reduce delays 
due to traffic congestion, reduce fuel costs, increase the amount of throughput to two to three times 
the amount of current day traffic, and create technology that will allow air traffic controllers to 
manage airspace traffic safer and more efficiently. 

The air traffic management system used in current day operations will be unable to manage this 
growth. In 2011 alone, out of 3,567,652 flights, 20.46% were late arrivals, 19.13% were late 
departures, and 2.44% were cancelled, with only 76.83% of arrivals on time (Bureau of 
Transportations Statistics, 2012). These amounts of delays in the air traffic management system 
show how the transition from current day operations to the NextGen is vital for efficient airspace 
operations (JPDO, 2012a). 

Fundamental changes in many of the technologies and procedures used today are projected to be 
seen in air traffic automation, communications and navigation for accommodation of the expected 
two to three time increase in airspace traffic in the NAS (FAA, 2012b). As the NextGen airspace 
system will be required to do increasingly more tasks, the roles and responsibilities presently 
performed by the airspace management workforce will be required to change along with these new 
tasks. Research is being performed to develop decision support tools for both the pilots of the 
aircraft and the air traffic controllers, including testing of cockpit situation displays for use by pilots, 
as well as new merging and spacing procedures and algorithms (JPDO, 2011). 

The increase in traffic density and change in the roles and responsibilities will create effects on the 
airspace users, such as an increase in workload and a reduction of availability in air traffic 
controllers (Galster, Duley, Masalonis, & Parasuraman, 2001). One proposed measure to reduce the 
amount of controller workload, increase their availability and to increase the amount of throughput 
per sector is the use of delegated separation. 

1.2 Delegated Separation 

Delegated separation is the transfer of responsibility for maintaining separation between aircraft or 
airspace vehicles from air navigation service providers to the relevant pilot or flight operator (JPDO, 
2010). The high level tasks associated with separation assurance are the identification of potential 
conflicts and losses of separation, the identification of a solution for the problem (how the aircraft 
should maneuver to avoid conflicts), the implementation of the solution (change in heading, speed, 
altitude, etc.), and then monitoring for clear of conflict (Hoffman, Zeghal, Cloerec, Grimaud, & 
Nicolaon,1999). 

Tasks associated with separation assurance are then delegated out to the aircraft pilot or flight 
operator in delegated separation. Three major levels of delegation have been defined: limited 
delegation, extended delegation and full delegation (Zeghal, & Hoffman, 2000). The exact tasks and 
responsibilities associated with delegated separation may vary depending upon airspace type, aircraft 
type, and phase of flight (Domino, Tuomey, Mundra, & Smith, 2010). When operating in oceanic 
airspace, delegated separation may be used for overtaking another aircraft. While in terminal areas, 
aircraft may be performing a longitudinal station keeping task or perform traffic merging. When 
operating in an airport, pilots may be performing runway incursions or avoiding obstacles while 
maneuvering on the ground. 
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Limited delegation places the controller in charge for identifying both the potential conflict and the 
appropriate solution. Pilots are responsible for implementing the solution and monitoring for their 
ownship to be clear of conflict so that they can request permission to return to their original flight 
path. Extended delegation places the controller in charge of identifying and notifying the pilot of 
potential conflicts. The pilot is then responsible for identifying the appropriate solution, 
implementing the solution, monitoring for clear of conflict, and requesting permission from the 
controller to return to their original flight path. In full delegation, the air traffic controller assumes 
more of a monitoring role as the pilot assumes responsibility for all tasks related to separation 
assurance: identification of conflicts and appropriate solutions, implementation of the solutions, 
monitoring for clear of conflict, and resuming the original flight path (Zeghal, & Hoffman, 2000). 

A concern with using delegated separation in the NextGen airspace environment is the possibility for 
fundamentally changing the air traffic controller roles and responsibilities, potentially reducing the 
controller’s situation awareness of the sector in their command and creating a change in their 
workload. By handing off separation responsibilities to the aircrafts’ flight crews, the controllers’ 
task changes from being one of active control to a task predominately characterized by monitoring 
the aircraft in their sector (Galster, Duley, Masalonis, & Parasuraman, 2001). The additional 
responsibilities to be delegated to pilots and flight crew cause concern as well. Pilots are not 
currently trained on traffic management (FAA, 2008a), and the amount of traffic information 
required to make successful deviations for avoiding conflicts needs to be researched further. 

While the changing roles and responsibilities of air traffic controllers and pilots are of concern, the 
expected benefits of delegated separation are numerous. On the air traffic controller side, a reduction 
in workload, a reduction in the amount of radio communications between aircraft and controllers, 
and a reduction in the amount of interventions by controllers are expected. A reduction in the 
amount of maneuvers performed by the aircraft is anticipated to also create a decrease in the amount 
and cost of fuel used per flight, increased traffic throughput per sector, and an increase in pilot and 
flight deck crew situation awareness (JPDO, 2012b). 

1.2. 1 Current Day Delegated Separation 

While delegated separation is viewed as being a largely NextGen concept, varying types of 
delegated separation are allowed in Class B, C, D, and E airspace in current day operations. Cockpit 
display of traffic information (CDTI) assisted visual separation allows pilots to maintain visual 
approach spacing even if visual contact is lost (JPDO, 2012b). This capability is incredibly useful in 
situations with low or poor visibility, such as stormy weather or nighttime flights. Since 2007, UPS 
has been performing CDTI assisted visual separation out of Lousiville, KY (Henden, 2008). 

Visual separation is described in FAA Order JO 71 10.65 U (2008b).There are two ways to perform 
visual separation; the controller identifies a conflict and issues instructions to the pilot as necessary, 
or the pilot identifies a conflict aircraft, calls in to air traffic control, and provides their own 
separation by maneuvering their aircraft as necessary. When a conflict is identified the air traffic 
controller will typically give the pilot information on the position, direction and intention of the 
intruding aircraft. If the pilot notices the traffic first, the notification process may be reversed with 
the pilot notifying the controller of the intruder. After the pilot acknowledges the intruding aircraft is 
visually in sight, the controller will instruct the pilot to maintain visual separation. The pilot will 
then accept the order to maintain visual separation, and maintain separation from the aircraft. 
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1.2.2 Delegated Separation in NextGen Airspace 

The use of delegated separation in NextGen airspace has been addressed by both the FAA and the 
Joint Planning and Development Office (JPDO) as part of Automatic Dependent Surveillance- 
Broadcast (ADS-B) uses. ADS-B consists of two parts, ADS-B In and ADS-B Out, and will be used 
as the primary surveillances method in NextGen airspace instead of radar (FAA, 2010). ADS-B Out 
broadcasts real- time information on aircraft state, such as identification information, altitude and 
velocity through a transmitter located onboard the aircraft. ADS-B In receives the information sent 
from ADS-B Out, not only from the ownship but also from nearby equipped aircraft, and has the 
potential to receive additional transmitted data. In many cases, the data provided by ADS-B is more 
accurate than radar information, and will allow the aircraft to perform more precise maneuvers for 
spacing and conflict avoidance. 

ADS-B Out will be required for aircraft operating in most of the NAS by 2020. With the precise 
information provided by ADS-B, an increase in traffic situation awareness and better conflict 
detection alerting will be possible. Through the use of ADS-B In, new spacing and separation 
capabilities will be possible starting as soon as 2017 (FAA, 2012c). Flight deck based interval 
management spacing is a capability to be used for managing spacing between aircraft while flying 
enroute. CDTI assisted visual separation is an emerging use of ADS-B data that is planned for 
expansion. It allows pilots to maintain visual approach spacing even if visual contact is lost, such as 
in situations with stormy weather or nighttime low visibility. 

The concept of delegated separation through use of ADS-B data is addressed as well. The defined 
interval concept discussed in the FAA NextGen Implementation Plan (2012b) is an example of 
extended delegation. In a defined interval task, the air traffic controller is seen to hold responsibility 
for maintaining separation while assigning pilots a spacing task that will be performed within 
defined boundaries. This use of pilot assigned spacing is believed to create a closer baseline for 
interval spacing than is currently possible through use of radar information. Delegated separation is 
again suggested through the Oceanic in-trail procedure (ITP) where a climb-through or descend- 
through maneuver is initiated by the flight crew of equipped aircraft in conflict. While performing 
this maneuver, pilots are to maintain the in-trail minimum separation. A full description can be seen 
in RTCA DO-312/EUROCAE ED-159 (RTCA, 2008). 

The JPDO provides NextGen planning organizations with a view of mid-late and far-term (2025) 
operational capabilities expected to be in use in the NAS. Much of the work done by the JPDO is 
expected to be incorporated into the FAA NextGen Implementation Plan in the succeeding years as 
the FAA focuses on the near term aspects of a NextGen implementation. As part of the JPDO’s 
NextGen Avionics Roadmap (2011), delegated separation is seen to provide an enhanced situation 
awareness that is shared between pilots in the air, and the controllers on the ground. This enhanced 
situation awareness will allow delegated separation practices to expand from those currently used in 
visual conditions, to non visual conditions in controlled airspace. 

1.3 Unmanned Aircraft Systems 

Unmanned aircraft systems are more than just an aircraft with the crew removed and placed on the 
ground. UAS are typically comprised of the aircraft (also known as an unmanned aerial vehicle or 
UAV), payloads, launch and recovery subsystems (when applicable), support sub-systems, the 
control station(s), communication devices (typically radio), vehicle operator, sensory operator(s), 
and support crew (Gertler, 2012). The system elements used to operate UAS are typically based 
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upon those used in manned aircraft, though slightly different as they are created with the knowledge 
that the aircrew will not be onboard. 

When talking about UAS, it is important to denote the differences between UAS, model aircraft and 
drones as they are easily confused (Austin, 2010). Model aircraft are typically designed to be 
operated for recreation, and restricted to use in sight of the operator through their command link. 
Drone aircraft typically contain no “intelligence,” and no capabilities of sending or receiving any 
mission-pertinent information while in flight such as photographs, signal readings, or even vehicle 
location. Drones will often have a pre-planned flight path with a payload collecting data throughout 
their mission. This data is not accessible until after the drone is recovered and the operator removes 
the payload. 

Unmanned aircraft systems can contain varying levels of intelligence, and are capable of 
communicating with the operator information such as payload data, flight state information 
(position, airspeed, heading, and altitude), and vehicle state information (amount of fuel, engine 
status, vehicle temperature, etc.) throughout their mission task. More intelligent UAS, such as the 
Global Hawk, can even be preprogrammed to perform contingency actions when specific off- 
nominal events, such as a loss of command and communication signal, occur (Mouloua, Gilson, & 
Hancock, 2003). 

1.3.1 Types of Unmanned Aerial Vehicles 

While UAS have many components besides the vehicle itself, the systems are typically categorized 
by the body type (such as fixed wing, turbo prop, or rotorcraft), size and capabilities (such as vertical 
takeoff and landing) of the vehicle used in the mission (Special Committee 203, 2010). The 
categories and specifications used to identify UAS can vary depending upon the agency, though a 
generic break down of UAV types may be used to describe them. 

High Altitude Long Endurance, or HALE, UAVs can perform missions in high altitude (30,000+ ft), 
typically 24 hours or longer in duration. Typically HALE UAYs are used for missions that are long- 
range, require reconnaissance or surveillance, and have the potential to be armed and used for target 
acquisition and prosecution. The RQ-4 Global Hawk made by Northrop Grumman is an example of 
a HALE UAV (NASA, 2010). 



Figure 1 . Global Hawk UAS. Image reproduced with permission from NASA. 
Copyright 2010 by NASA. 


Medium Altitude Long Endurance (MALE) UAVs can perform missions from up to 30,000 ft in 
altitude and within 24 hours or less in duration. MALE UAVs are similar in many ways to HALE 


5 



UAVs, and are often operated in a similar manner. An example of a MALE UAV is the MQ-9 
Predator B made by General Atomics (NASA, 2007). 



Figure 2. MQ-9 Predator B UAS. Image reproduced with permission from NASA. 
Copyright 2007 by NASA. 


Tactical or Medium Range UAV (TUAV), are smaller than MALE UAVs, and operate on simpler 
systems than the larger vehicles. TUAVs consist of both fixed wing and rotorcraft vehicles, with 
some aircraft having the capability to take off from and land on runways and airstrips. Aircraft may 
be accompanied by a ramp launching system to shoot the aircraft into the sky at flight speed. 
Examples include the Hunter RQ-5A made by Northrop Grumman, and the Shadow 600, made by 
AAI Unmanned Aircraft Systems (AAI Unmanned Aircraft Systems, 2009). 



Figure 3. Shadow ® Tactical unmanned aircraft system. Image reproduced with 
permission from AAI Unmanned Aircraft Systems. Copyright 2009 by 
AAI Unmanned Aircraft Systems. 


Close Range UAVs are a prolific subset often used by groups that do not have a set location and are 
moving due to their smaller size. As Close Range UAVs are often used in locations that are lacking 
in runways and airstrips, they are often accompanied by a launching system that typically uses a 
ramp mounted on a vehicle used in transport to shoot the aircraft from the ramp at flight speed, 
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similar to those used by some TUAVs. Recovery of the Close Range UAYs is typically done 
through use of a parachute and airbag combination to lessen the damage of a fall, though methods 
may vary such as in the case of the sky-hook used for the Scan Eagle. Their use can range from 
reconnaissance, and target acquisition, to power line inspection, and crop spraying. Examples 
include the Scan Eagle made by Insitu, and the Aerosonde Small Unmanned Aircraft System made 
by Aerosonde Pty Ltd, (2012). 



Figure 4. Aerosonde ® small unmanned aircraft system. Image reproduced with 

permission from AAI Unmanned Aircraft Systems and Aerosonde Pty Ltd. 
Copyright 2012 by Aerosonde Pty Ltd. 


Mini UAV (MUAV) is a class of UAVs that is not yet clearly defined. MUAVs are generally hand 
launched, and operated through the use of a laptop ground control station. Like Close Range UAVS, 
MUAVs are often used by groups that are mobilized due to their small size and portability. 

Examples include the Desert Hawk III made by Lockheed Martin, and the Skylite made by BlueBird 
Aero Systems. 

Smaller UAVs have been created for use in urban terrains and research. The smaller sizes of these 
UAVs allow the vehicles to fly inside urban areas as opposed to flying over them. A desire for 
UAVs being used in urban terrains is to have the capability to hover and potentially perch on walls 
or window sills, creating unconventional configurations not commonly seen in the larger UAVS, 
such as the use of flapping wings (ornithopeters). 

The Micro UAV (MAV) was initially defined as having a wing span of less than or equal to 6 
inches, though current definitions may vary. MAVs are preferred for use in urban terrains as their 
small size and maneuverability make them ideal for navigating between and within buildings. 
Examples include the Wasp made by AeroVironment Inc, and the Mosquito made by Israel 
Aerospace Industries. 

Nano Air Vehicles (NAV) is an emerging category populated by small vehicles less than 5 cm in 
any direction. The “nano” in the name comes from the requirement that a vehicle so small will need 
nanotechnology for use in subsystems such as batteries, sensors and motors. NAVs may be used in a 
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similar manner as MAVs, though they are also expected to be used in swarms, possibly for radar 
confusion or short range surveillance as most operate for only a few minutes at a time due to their 
small size. Examples include the Prox Dynamics Pico-flyer, Ornithopter, and Black Hornet (Prox 
Dynamics, 2012). 




Figure 5. Nano Air Vehicles. Examples of NA Vs include the Pico-flyer ( top left), the 
Ornithopter (top right), and the PD-100 Black Hornet (bottom). Image 
reproduced with permission from Prox Dynamics. Respective copyrights 
2005, 2007, and 2009 by Prox Dynamics. 


Some of these categories may utilize rotorcraft vehicles, often called Remotely Piloted Helicopter 
(RHP), Vertical Takeoff UAV (VTUAV) or Vertical Takeoff and Landing (VTOL) UAV. These 
UAVs are valued for their ability to hover during missions, and for being less susceptible to 
turbulence than fixed wing aircraft. It is of interest to note that not all aircraft that are capable of a 
vertical takeoff are capable of vertical landing. Some examples include the Firescout made by 
Northrop Grumman, and the Camcopter made by Schiebel. 

Combat UAS are split into two main categories. Unmanned Combat Aerial Vehicles (UCAV) are 
fixed wing aircraft designed to carry and launch weapons which may be used in aerial combat 
situations. Unmanned Combat Rotorcraft (UCAR) are also in development. Examples include the X- 
45A made by Boeing, and the X-47 Pegasus made by Northrop Grumman (NASA, 2002). 



Figure 6. The X-45A UCAV. Image reproduced with permission from NASA. 
Copyright 2002 by NASA. 
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1.3.2 Civilian UAS Use 


Unmanned systems are often used for duties that are deemed to be too dull, dirty or dangerous for 
human completion (Takayama, Ju, & Nass, 2008). In addition to the known military use of UAS for 
training, transit, surveillance, and reconnaissance, there is a wide variety of civil applications. 
Proposed uses include research, disaster monitoring and aid, agriculture, surveillance, and 
conservation among others. 

Research applications include geological and meteorological aspects. Meteorological applications 
often include the sampling of atmospheric components that can be used to further understand 
weather phenomena and create more accurate weather forecasting. Weather monitoring data can 
include information on air temperature, dew point, atmospheric pressure, winds, global warming 
monitoring, and atmospheric pollutions ratings (NOAA, 2012b). 

Currently UAS are utilized for research missions, including the Global Hawk used by NASA and 
National Oceanic and Atmospheric Administration (NOAA) primarily focused on collecting 
information about storm development and tracking (NOAA, 2012a). This Global Hawk utilizes a 
special payload called the DropWindSonde System (Figure 7) designed to be dropped into the top of 
the storm and continuously collect data until the weather dissipates (NOAA, 2012b). 



Figure 7. The NOAA DropWindSonde system probe. Image reproduced with 
permission from NOAA. Copyright 2012 by NOAA. 


The use of UAS for disaster monitoring and aid is done through the use of video and infrared search 
within specific restrictions defined by the FAA (Aviation Unmanned Aircraft Program Office, 
2008). The first time that UAS were requested for search and rescue operations was in 2005, in the 
wake of Hurricane Katrina (Waharte & Trigoni, 2012). At the time, the FAA did not have any 
regulations in place for this type of UAS use, and no way to authorize the mission for flight in the 
national airspace. Regulations have since been passed to grant authorization for relief missions in a 
matter of hours. 

The MQ-1 Predator is now authorized to fly in search and rescue missions in support of disaster 
relief as the UAS has an infrared camera that is able to identify heat source emissions small 
enough to come from a human body up to approximately 10,000 ft away (Waharte, & Trigoni, 
2012). This infrared camera can assist rescuers to identify and locate survivors quickly, as well as 
increase the efficiency of search patterns and crew organizations. The infrared camera is especially 
useful during wildfire flares, and has been used to assist firefighters in locating hidden pockets of 
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fire and mapping the movement of fires across extended periods of time (NASA, 2007). An 
infrared image created by the Ikhana Predator of the Harris Fire in San Diego County (2007) can 
be seen in Figure 8 . 



Figure 8. UAS wildfire infrared imaging. Thermal infrared image scans from NASA's 
Ikhana Predator UAS of the Harris Fire in San Diego County October 24, 
2007 . Image shows wildfire hot spots in yellow along the ridgeline. Image 
reproduced with permission from NASA. Copyright 2007 by NASA. 


Agricultural and fisheries uses are varied and can include crop monitoring and spraying, livestock 
monitoring and herd driving, fisheries protection and water stress, as well as habitat monitoring. 
Using UAS for these purposes creates a number of potential advantages, such as with the cost of 
aerial imagery. Aerial imagery is typically collected from either satellite data, which is often not up 
to date and can often have a low resolution, or from data collected by having a light aircraft fly over 
the specified lands. Both options are relatively expensive, and the use of Micro UAS imaging could 
potentially cut down the cost of acquiring the imagery (Austin, 2010). The FAA currently does not 
allow the use of UAS for agricultural or fisheries purposes, though other countries have used them 
for years. Farmers in Japan, for example, have used small UAS for approximately 10 years to 
perform monitoring and crop spraying tasks. 

While surveillance and monitoring is typically associated with military tasks, there are many civil 
uses. Aerial photography and video recordings are used in a variety ways from land surveying, 
power and gas line inspections, oil pipeline security, news and media use, monitoring and control of 
traffic on roads and highways, and ordinance surveys for creation of detailed maps. Perhaps one of 
the more well known surveillance and monitoring tasks UAS are used for is police and customs 
work. The UAS are capable of searching for mission persons and suspects, checking coastline and 
land bound borders for illegal immigration suspects, illegally imported goods and search and rescue 
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missions at sea (DoD, 2011). Currently both the U.S. Border Patrol and the U.S. Coast Guard have 
multiple UAS in use, including the fixed wing Predator B and the Eagle Eye tilt rotor helicopter. 

There also exists the capability for UAS to assist in conservation efforts. Through the use of UAS, 
conservationists are able to monitor atmospheric and land based pollution, forestry services are 
capable of identifying and controlling fires, monitoring deforestation patterns, and wildlife 
monitoring. The use of UAS is especially helpful when observing endangered or threatened species 
that have extensive migratory patterns, or live in hard to access regions. UAS have been successfully 
used in conservation efforts for estimating shrub utilization, identifying and locating invasive 
species, measurement of the biomass and nitrogen levels present in various plan species, 
identification of crop water stress, and mapping various rangeland plant species (Austin, 2010). 

1.4 Unmanned Aircraft System Integration into the National Airspace 

The integration of UAS into the NAS is seen to be one of the critical issues existing between now 
and the mid-term vision of NextGen (JPDO, 2012b). The demand for public access of UAS in the 
U.S. National Airspace (NAS) has grown dramatically. Projections of development from 2010 to 
2019 predict over 20,000 UAS created in the United States and 35,000+ created worldwide (Teal 
Group, 2009). The FAA Modernization and Reform Act of 2012 addresses this increasing demand 
by requiring full integration of UAS into the NAS by September 30, 2015 (FAA, 2012a). 

As part of this bill, the FAA is required to create a plan in order to successfully integrate UAS into 
the national airspace without compromising the safety and efficiency of the existing air traffic 
management system and its users (FAA, 2012a). The plan is required to have at minimum 
recommendations on how to define acceptable operational standards for flight and certification of 
UAS. The purpose of this section of the bill is to ensure that all UAS will have a sense and avoid 
system, and to create the standards for what would be required to act as a UAS operator in the NAS 
as well as the process to achieve certification. 

1.4.1 Sense and Avoid 

Sense and avoid is the capability to maintain separation from intruding aircraft. Sense and avoid 
systems may include a suite of surveillance sensors, trackers, threat detection and/or resolution 
algorithms, a traffic display for the pilot, and potentially resolution guidance or advice (Prinzel, et. 
al., 201 1). The addition of a sense and avoid suite on a UAS would create the ability for the UAS to 
avoid collisions through a combination of self-separation and collision avoidance. 

The capability for a UAS to self-separate is deemed to be an essential component of the sense and 
avoid system, and may be the only function required in the sense and avoid system provided a safety 
analysis can demonstrate the target level of safety is met (JPDO, 2012b). While operating in 
positively controlled airspace, separation responsibilities may be delegated to the UAS operator. 
Conflict avoidance alone on a UAS is not an acceptable means to remain well clear as stated in 
14CFR Part 91 (FAA, 2001); even if the target level of safety was met, a self separation capability 
would be required. 

1.5 Challenges and Concerns of UAS Integration 

The NAS was originally designed for use of manned aircraft, and while many procedures and 
general principles from manned aircraft can apply for UAS, there are significant differences is 
capabilities, advances in technology, and operational experience. These performance differences 
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between UAS and manned aircraft may cause disruption in the NAS . Four challenges have been 
identified for UAS integration into the NAS; communication, airspace operations, internal systems 
onboard the aircraft, and human systems integration (JPDO, 2012b). 

Communication concerns exist about the transfer of information between the aircraft, the operator, 
and the satellite/technology providing communications relay. Currently, communications relays can 
take on average two to eight seconds to send information from the aircraft to the operator. In a time 
sensitive situation, such as when avoiding a collision, this delay may be the difference between a 
crash and successful conflict avoidance. The security of the communications link is also of great 
concern as this could be hacked into, and control of the unmanned vehicle lost (DoD, 2001). 

Airspace operations concerns are associated with the UAS operating in the same airspace as manned 
aircraft. The quality and availability of surveillance data needs to be assessed. Integrated separation 
concepts need to be developed, requiring an evaluation of performance of different potential human 
and machine roles and responsibilities, and how to properly integrate the varying self separation, 
separation assurance and conflict avoidance functions. With the need for a sense and avoid system 
capable of self-separation, the differences in unmanned and manned aircraft flight become points of 
note. Concerns exist about UAS performing self-separation due to the lack of the out-the- window 
view that manned aircraft pilots have (Gawron, 1998). Without this visual means of traffic 
acquisition, a sense and avoid solution becomes vital for safe UAS flight. UAS self-separation is an 
essential component of the sense and avoid solution, and may be the only function provided safety 
analysis demonstrates the target level of safety is met (JPDO, 2011). 

The unmanned aircraft itself, and systems located onboard, are also items of concern. Airframe 
certification, location and navigation systems, UAS avionics and control systems certification, as 
well as the ability for the UAS operator to have an accurate awareness of vehicle state and be 
capable of real time management are all items that need to be further researched (McCarley, & 
Wickens, 2005). The flight characteristics of UAS are different from manned aircraft in many ways, 
such as being operated at speeds slower than those typically used for manned aircraft and being able 
to fly at higher altitudes than many manned aircraft. 

The overall flight paths of manned and unmanned aircraft are also quite different and important to 
note . Manned aircraft typically aim to fly the shortest distance from Point A to Point B through the 
use of specified routes and arrival corridors. Unmanned aircraft are often required to operate in a 
grid or corkscrew pattern, loiter, hover, and perform frequent heading and altitude changes, as well 
as unplanned aerial work around areas of interest while performing mission roles (Special 
Committee 203, 2010). An example of a UAS flight path can be seen in Figure 9. The differences in 
mission behavior have the potential to cause congestion and an increase of potential conflict in 
sectors as the flight path of a UAS may appear to be erratic and unpredictable when compared to that 
of a manned aircraft. The differences in flight path configuration and mission roles are important to 
take into consideration as they may increase the workload of air traffic controllers managing the 
sector the UAS is flying in. 

Human system integration concerns focus on the interaction between the operator and the ground 
control station’s interfaces and systems (JPDO, 2012b). Currently there is no universal ground 
control station used for unmanned aircraft, nor are there any guidelines for what should be required 
in a ground control station, or of the UAS operator. Research must be conducted to determine the 
optimal performance needs. Areas needing research include: the display of traffic, aircraft and 
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airspace information, focusing on the presentation method and quantity of information available; 
how to promote optimal human and automation interaction; system level issues for any interfaces or 
algorithms being utilized; a pilot centric station that includes an ergonomic evaluation for optimal 
placement of displays and controls; a clear definition of roles and responsibilities given to the UAS 
operator and the air traffic controller; and a set of qualification and training requirements needed to 
operate safely in the NAS (DoD, 2011). 



Figure 9. UAS flight path example. This image shows the planned route of a 

ScanEagle UAS in a green corkscrew pattern performing a marine fisheries 
protection and monitoring operation. Image reproduced with permission 
from RTCA, Inc. Copyright 2010 by RTCA, Inc. 4 


The NextGen airspace will be required to provide a NAS that is flexible and robust enough to 
support routine use of UAS instead of as the exception. This could be accomplished through a 
combination of automation, more precise airspace information provided by ADS-B, and delegated 
separation given to the pilot within positively controlled sectors (Special Committee 203, 2010). It is 
theorized that much of the activity would occur in air traffic controller systems using automation, 
and on systems located in manned aircraft and unmanned ground control stations to advise pilots of 
relevant flight path changes. This will help to maximize airspace efficiency and safety. 

Through use of self-separation, aircraft exchanges would be accepted or rejected based upon the 
situation awareness of the UAS operators and manned pilots. The role of the air traffic controller 
would be changed into one predominated by monitoring, with intervention only when necessary 
(Grimaud, Hoffman, & Zeghal, 2001). Self separation is planned to initially take place in areas with 
low traffic density, with a gradual progression to high density airspace. The UAS flying in NextGen 
airspace could benefit greatly from delegated separation as they will also begin to operate in low 
density areas and perform flight paths and mission tasks in a manner that is not typical for manned 
aircraft (Special Committee 203, 2010). The use of delegated separation by UAS operators could 


4 The complete document referenced, DO-320 Operational Services and Environmental Definition (OSED) 
for Unmanned Aircraft Systems (UAS), may be purchased from RTCA, Inc. 1 150 18th Street NW, Suite 910, 
Washington, DC 20036. (202) 833-9339 www.rtca.org. 
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reduce the amount of workload placed on air traffic controllers when monitoring for unexpected 
maneuvers and aircraft flying atypical routes. 

1.6 The Current Experiment 

The purpose of the current experiment is to determine the feasibility of UAS performing delegated 
separation in the NAS. The study was a 2 x 2 within-subjects design, with level of delegation 
(extended delegation vs. full delegation) and traffic display information level (basic, traffic only vs. 
conflict detection present) as independent manipulations. The effects of delegated separation and 
traffic display information level were assessed through objective and subjective measures of 
performance, workload and situation awareness. Experimental data were collected in the Flight Deck 
Display Research Laboratory (FDDRL) at NASA Ames Research Center from the experimental 
participants consisting of 13 pilots and two retired air traffic controllers acting as pseudo-controllers. 
Three pseudo- pilots were used in a confederate role to create sector traffic with no data collected on 
their performance. 

The expected results for air traffic controllers in the full level of delegation include: reduced 
workload, radio communications with the UAS operator to be perceived as easier, and less frequent 
communications with the UAS pilot. The expected results for the UAS operator include an increased 
(but manageable) workload in the full level of delegation, and increased situation awareness within 
both the full level of delegation and the conflict detection present mode of the CSD. It is expected 
that both the air traffic controller and the UAS pilot will have higher levels of workload in extended 
delegation, and subjective scores rating their perception of difficulty in communication will increase 
as well. The amounts of losses of separation are expected to be lower in full delegation and in the 
conflict detection present CSD mode. The overall amount of losses of separation levels are expected 
to be low. 


2.0 METHOD 

2.1 Participants 

2.1.1 Experimental Participants 

A total of 15 experimental participants were used for this study; 13 pilots acting as UAS operators, 
and two retired air traffic controllers acting as pseudo controllers. Data was collected from the 13 
pilots and the two pseudo controllers. UAS operator participation was limited to pilots who were 
between 18-40 years old, had normal or corrected to normal vision, and held at minimum a Private 
Pilot Certificate with preference towards those with an Instrument Rating. Pseudo controller 
participation was limited to retired air traffic controllers with previous experience using the Mutli- 
Aircraft Control System (MACS) controller mode software (AOL, 2008). Each air traffic controller 
worked with approximately half of the UAS operator participants throughout the simulation. 

2. 1.2 Confederate Participants 

Three pilots acted as confederate pseudo pilots to create the background airspace traffic and radio 
chatter. Pseudo-pilot participation was limited to pilots who were between 18-40 years old, had 
normal or corrected to normal vision, held at minimum a Private Pilot Certificate and had previous 
experience using the pseudo-pilot mode of MACS. No data was collected from the pseudo pilots as 
they were acting in a confederate role. 
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2.2 Experimental Design 

The current study was a 2 x 2 within-subjects design with level of delegation (extended delegation 
vs. full delegation) and traffic display information level (basic— traffic only vs. conflict detection 
present) as independent variables. Counterbalancing was used to control for order effects across the 
four experimental scenarios created as the result of the factorial combination of the two levels of 
delegation and two levels of traffic display information. The scenarios were blocked by traffic 
display condition; the first traffic display condition was presented with the two levels of delegation 
before the second traffic display condition was presented with the two levels of delegation. Order of 
presentation of the traffic display condition (basic or conflict detection) was counterbalanced across 
participants. All experimental data was collected in the Flight Deck Display Research Laboratory 
(FDDRL) at NASA Ames Research Center from the participants consisting of 13 pilots, two retired 
air traffic controllers acting as pseudo-controllers, and three pseudo-pilots. 

Data on the effects of delegation and traffic display information level was collected through 
objective and subjective measures of performance, workload and situation awareness. The level of 
delegation manipulation was used to uncover the effects of delegating the roles and responsibilities 
for maintaining separation partially or fully from the air traffic controller to the UAS operator. The 
level of traffic display information was used to uncover the effects of increasing information 
available about the surrounding airspace on operator performance when delegated with separation 
responsibility. As there were only two air traffic controllers in this experiment, it is important to note 
the analyses performed on their data have low statistical power. 

2.2. 1 Level of Delegation 

The level of delegation was separated into two conditions, extended delegation and full delegation, 
which were counterbalanced across missions. In both scenarios, only the UAS was given delegated 
separation and the air traffic controllers maintained positive control of all surrounding aircraft. At 
any point in time throughout the scenarios, the UAS Operator was allowed to request help from the 
air traffic controller by giving the direction, flight level and callsign of the intruder aircraft. 

Example of a UAS Operator requesting help: 

• UAS Operator: “L.A. Center, this is PD-1. Traffic north bound, FL 280, callsign 
SWA242. Cannot perform separation, please advise.” 

• ATC: “PD-1, traffic acquired.” 

- ATC orders a reroute to avoid collision. 

In extended delegation, the air traffic controller was responsible for problem identification and 
notifying the UAS operator of the potential conflict. The UAS operator then located the intruder 
aircraft on their traffic display and told the air traffic controller that the traffic was acquired. The 
controller would then tell the UAS operator to maintain separation from that aircraft. The UAS 
operator then accepted the separation order, and was responsible for identification and 
implementation of conflict solutions and monitoring for clear of traffic after rerouting. 

Extended Delegation example: 

• ATC: “PD-1, traffic SWA749 at 2 o’clock, north bound, FL 280. Advise tracking.” 

• UAS Operator: “PD-1, traffic acquired.” 

• ATC: “PD-1, maneuver to maintain separation from SWA749.” 
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• UAS Operator: “PD-1, maintaining separation with SWA749.” 

- UAS makes a reroute to avoid collision and monitors until clear of conflict. 

• UAS Operator: “L.A. Center, this is PD-1. Clear of traffic, request to return to original 
flight path.” 

• ATC: “Roger, PD-1. Permission to return to original flight path approved.” 

- UAS returns to original flight path. 

Full Delegation gave responsibility to the UAS operator for all tasks related to separation assurance, 
including identification of problems and solutions, implementation of solutions and monitoring for 
clear of traffic after rerouting. Air traffic controllers only reclaimed separation responsibilities from 
the UAS operator if collisions became imminent or the operators requested assistance .No 
communication with air traffic control was required to complete these scenarios. 

Full Delegation example: 

• UAS Operator monitors CSD for potential conflicts. 

• UAS Operator identifies a potential conflict. 

• UAS Operator makes a reroute to avoid collision and monitors until clear of conflict . 

• UAS Operator identifies clear of conflict and resumes original flight path. 

2.2.2 Level of Traffic Display Information 

The level of traffic display information was split into two conditions, basic and conflict detection 
present, and counterbalanced across missions. The basic conditions provided a 2-D display of the 
surrounding traffic in an ownship centric manner. Basic information included aircraft call sign, 
altitude and airspeed, as well as the use of color-coding to denote altitude (green for 500+ ft below 
ownship, white for within 500 ft below or above ownship, and blue for 500+ ft above ownship). No 
conflict detection was available in the basic conditions. The conflict detection present conditions 
included the information available in the basic conditions and provided conflict detection alerts in a 
visual and aural manner that was based upon ballistic information. Once the UAS operator 
successfully rerouted to avoid the conflict, the ownship and intruder aircraft stopped glowing yellow. 

2.2.3 Missions 

Two training and four experimental scenarios were used in this experiment. The training scenarios 
were five minutes long and provided practice for all participants. The UAS operators learned how to 
operate their aircraft and how to use their ground control station to perform flight path reroutes while 
practicing under the assigned level of delegated separation. Air traffic controllers practiced 
delegating separation responsibility to the UAS operator in the assigned level of delegated 
separation. 

Experimental scenarios were each 30 minutes long, and consisted of a C0 2 emissions monitoring 
task. No conflicts were pre-scripted, though the flight paths of the UAS throughout the trials were 
designed to cross over and through traffic streams so as to increase the chance of a conflict 
occurring. All scenarios began with the UAS in a preprogrammed flight path and included five 
mission messages that required the UAS operators’ attention and flight path reroutes. The timing of 
the mission messages and the four flight paths differed between scenarios and were counterbalanced 
as a means to reduce predictability. 
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2.2.4 Mission Objectives 

The UAS Operators were instructed to fly a C0 2 emissions monitoring task in southern California 
based upon scenario 3 in RTCA SC-203 ’s OSED for UAS (2012). The mission objectives that were 
given to the UAS operator included: follow the appropriate level of delegated separation while 
flying the mission routes, reroute in response to mission messages, reroute to maintain separation 
from the surrounding traffic, and maintain communications with ATC as necessary. Air traffic 
controllers were also informed of the level of separation delegated to the UAS operator, and notified 
of the corresponding responsibilities in each of the scenarios. Air traffic controllers always 
maintained positive control over the surrounding aircraft; only the UAS received delegated 
separation responsibilities. 

2.3 Experimental Environment 

The experimental environment was a simulated LA Center airspace. UAS operators and air traffic 
controllers had access to a list of currently used navaids and fixes for use in communications and 
route modifications. The current day air traffic controller will accept and hand off aircraft entering or 
exiting their sector and maintain separation between aircraft to avoid mid-air collisions. Separation 
standards for aircraft are typically a 5 nm mile horizontal distance, and 1000 ft in vertical distance in 
current day operations (FAA, 2008b). If the controller fails to maintain this level of separation, a 
loss of separation (LOS) occurs; collisions may or may not occur depending upon how far the 
intruder penetrates the separation standard. 

In half of the trials, separation responsibilities were partially delegated to the UAS operator, and in 
the other half separation responsibilities were fully delegated to the UAS operator. In all 
experimental conditions, the air traffic controller maintained positive control of the surrounding 
aircraft; only separation responsibilities for the UAS were delegated away from the controllers. The 
UAS operators used a ground control station composed of the U.S. Army Aeroflightdynamics 
Directorate’s Multiple UAS Simulator (MUSIM) (Fern & Shively, 2009) and the Ames 3D CSD 
(Granada, Dao, Wong, Johnson & Battiste, 2005) to display traffic. The air traffic controllers and 
confederate pseudo pilots used the MACS in their respective air traffic controller and pseudo pilot 
modes. 

All experimental participants additionally had a touch screen monitor used to collect workload 
ratings through probes. Workload probes were presented with an auditory chime to alert the 
participants of their presence. UAS operator probes were presented every three minutes with a scale 
from one (low) to seven (high), while air traffic controller probes were presented every three 
minutes on a scale from one (low) to nine (high). UAS operators were given six situation awareness 
probes per trial presented in a chat box that was located on MUSIM. Communications between 
participants and pseudo pilots were given by voice through a simulated radio frequency. 

2.4 Apparatus 

2.4. 1 Multiple UAS Simulator 

The MUSIM, shown in Figure 10, is a Linux based UAS ground control station simulation 
environment; a full description of its capabilities can be found in Fern & Shively (2009). The 
configuration of MUSIM for this experiment was in a 1:1 operator to vehicle interface with no 
sensor video as the focus was on the basic flight operation of the UAS and not sensor tasks. A 
generic fixed wing flight control model with generic Mid- Altitude Long Endurance UAS parameters 
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was used. Ownship speed for the UAS was fixed at 110 knots throughout all missions. Shutter 
screen capture software was utilized to record video of all events occurring on MUSIM. 



Figure 10. The MUSIM ground control station. The ground control station was 

separated into four sections. A map display used to indicate the position 
and flight path of the UAS, the available navaids for route modifications , 
and where the UAS operator performed route modifications (left). A multi- 
function display to display the UAS primary flight display and mission 
messages (top right). The mlRC Chat box used to display a simulated UAS 
chat group and the operator s situation awareness probes (middle right), 
and a timer (bottom right). 


2.4.2 The Ames 3D CSD 

The Ames 3D CSD (Figure 1 1) is a 3D volumetric display designed to provide pilots with an 
increased situation awareness of the surrounding traffic. A full description can be found in Granada, 
Dao, Wong, Johnson & Battiste (2005). 

In this experiment, the Ames CSD was used in its basic 2D planar view to display traffic 
information. The CSD has an ownship-centric view of the surrounding traffic that was color coded. 
Green aircraft were over 500 ft below the ownship’s altitude, white aircraft were within 500 ft above 
or below the ownship’s altitude, and blue aircraft were over 500 ft above the ownship’s altitude. 

Participants were allowed to adjust the horizontal viewing distance from 10-640 nm, and were 
provided with conflict detection alerts in half of the conditions. UAS operators were notified of 
traffic as far as 20 minutes into the future through alerts given in the form of an auditory chime, and 
a visual of their aircraft and the intruding aircraft glowing yellow on the cockpit situation display. 
Camtasia screen capture software was utilized to record video of all events occurring on the CSD. 
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Figure 11. The Ames 3D CSD. Green aircraft were 500+ ft below ownship’s 

altitude, white aircraft were within 500 ft above or below ownship’s 
altitude, and blue aircraft were 500+ ft above ownship’s altitude. The 
visual alert of the yellow glowing ownship and intruding aircraft can be 
seen indicating a conflict. 


2.4.3 Multi Aircraft Control System 

Multi Aircraft Control System is a JAVA program created by the Airspace Operations Laboratory 
(AOL) at NASA Ames for simulation of airspace to be used in air traffic management and 
operations research (AOL, 2008). In the current experiment, the MACS pseudo pilot (Figure 12) and 
air traffic controller (Figure 13) modes were used. The sector controlled by the air traffic controllers 
was between 13,000 and 24,000 ft, with the navaids and fixes that were available to the UAS 
operator displayed and boundaries denoted by highlighted lines. Camtasia screen capture software 
was utilized to record video of all events occurring on the pseudo pilots’ and air traffic controllers’ 
monitors. 
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Figure 12. MACS in pseudo pilot mode. MACS allowed the pseudo pilots to 

control multiple aircraft at a time through the use of automation . The 
monitor on the left displays the sector, and the monitor to the right 
displays aircraft controls. 



Figure 13. MACS in air traffic controller mode. MACS allowed the air traffic 

controllers to observe their sector and manipulate the traffic flow. The 
sector was between 13,000 and 24,000 ft, with navaids and fixes 
displayed, and boundaries denoted by highlighted lines. 
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2.5 Procedure 


After arrival in the lab, all participants were briefed on the purpose of the study, and given informed 
consent and demographic forms to complete. Participants were split into two groups; UAS operators 
and pseudo participants . The pseudo participants (including pseudo controllers and pseudo pilots) 
received their initial briefing and training before the experimental data was collected. This was done 
to reduce the amount of time spent on training as the pseudo participants rotated throughout the 
experiment. The UAS operator participants were briefed on the purpose of the study and provided 
with instructions on how to operate MUSIM and respond to online workload and situation awareness 
probes. 

UAS operator participants were then run through a five-minute practice scenario focusing only on 
the use of MUSIM and responding to probes without the presence of the pseudo controllers or 
pseudo pilots. After completion of the initial MUSIM practice scenario, participants were given 
training appropriate to their first experimental scenario block through the use of self-paced 
PowerPoint slides. After the training was completed, all participants completed a full five-minute 
practice scenario in the appropriate condition containing mission reroute messages and probes. 

After the training scenario was completed, participants then began their first 30 minute experimental 
scenario. After each experimental scenario, UAS operator and pseudo controller participants were 
given a NASA Task Load Index (TLX) to assess their level of workload. The experimental 
participants also responded to post-trial subjective questions asking about workload associated with 
communications and negotiations between the pseudo air traffic controllers and the UAS operators. 
After the questionnaires were completed, participants were given a 10 minute break. There were two 
delegated separation scenarios (extended delegation vs. full delegation) performed for each CSD 
experimental block (basic traffic information vs. conflict detection present). 

After each experimental block, the UAS operator participants completed a subjective questionnaire 
asking about situation awareness. After the first experimental block, the UAS operator participants 
were given the appropriate training for the next two scenarios through a self-paced PowerPoint slide 
presentation. The training was then followed by a five minute practice scenario with all elements 
associated with the experimental scenario given to the UAS operator, pseudo air traffic controllers 
and pseudo pilots. 

Post scenario and post block questionnaires were provided in the same order as given with the first 
experimental block. After the last scenario and associated paperwork was completed, the UAS 
operator and pseudo controller participants were given a post-simulation questionnaire asking 
questions on workload, situation awareness, pseudo controller to UAS operator interactions, as well 
as acceptability, preference and usability ratings. Participants were then debriefed, and the 
experimenter answered any questions the participants had. 

2.6 Metrics 

The effects of delegation and traffic display information level were assessed through objective and 
subjective measures of performance, workload and situation awareness. 

2.6. 1 Objective Metrics 

The objective metrics that were collected focused mainly on the UAS operator. Current separation 
standards are defined as having a minimum required distance of 1 ,000 ft vertical and 5 miles lateral 
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(FAA, 2008b). When aircraft are separated by a distance less than this, a LOS has occurred. 

Situation awareness probes were provided to the UAS operator through queries provided in the 
MUSIM chat client. Operators were required to monitor the chat window for their callsign (PD-1) 
and respond to any queries addressed to them. Responses were judged based upon the accuracy of 
the response, with the assumption that correct responses indicate high situation awareness. UAS 
operator response times were also analyzed. 

2.6.2 Subjective Metrics 

The subjective metrics collected included situation awareness ratings and multiple workload ratings 
from both the UAS operator and the pseudo controller. The UAS operators were provided with 
online workload probes every three minutes on a scale from one (low) to seven (high), and the 
pseudo air traffic controllers received online workload probes every three minutes on a scale from 
one (low) to nine (high). Both the UAS operators and the pseudo air traffic controllers were 
provided with a post-trial NASA TLX. 

The NASA TLX is a self- assessed workload measure that includes ratings on six dimensions; 
mental, physical, temporal, performance, effort and frustration (Hart, & Staveland, 1988). Along 
with the NASA TLX, both UAS operators and pseudo controllers were given post-trial and post- 
simulation questions pertaining to their overall workload associated with their interactions with each 
other (UAS operator to pseudo controller) on a scale from one (low) to seven (high), and additional 
information on UAS operator and pseudo-controller workload, situation awareness, and acceptability 
and preference ratings. 


3.0 RESULTS 

Multiple 2x2 repeated measures ANOVAs were performed on the data collected in this experiment. 
The repeated measures ANOVA is used to test the equality of means when the participants are 
exposed to all treatments (Howell, 2011). This test indicates a real difference between the means in 
the datasets compared to a difference found by chance or sampling error. 

ANOVAs were performed on the following data sets: 

• Loss of separation (within 5 nm horizontal and 1 ,000 ft vertical) 

• Pilot workload probes (ratings and response times) 

• Pilot NASA TLX (individual subscales and overall) 

• Pilot situation awareness probes (accuracy and response times) 

• ATC workload probes (ratings and response times) 

• ATC NASA TLX (individual subscales and overall) 

• Subjective questions on: 

- Difficulty of interaction between ATC and UAS operator 

- Acceptability of final flight path 

- Acceptability of traffic reroutes 

- Difficulty for ATC to maintain flow and separation with UAS present 

- ATC level of delegation preference 
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3.1 Loss of Separation 

Loss of separation data was analyzed by looking at the closest horizontal and vertical distances 
intruding aircraft reached in regards to the LIAS per trial. A loss of separation was determined to 
have occurred when the minimum distances reached between the UAS and the intruder were within 
a distance of 5 nm horizontal and 1 ,000 ft vertical. 

A 2x2 repeated measures ANOVA was used to determine if there were any significant differences 
between the level of delegation and the level of information shown on the traffic display. Results 
showed no significant differences between the basic and conflict detection present CSD condition, 
F(l, 12) = .274, p = .610, d= -0.327; no significant differences between the extended and full levels 
of delegation, F(l,12) = .085 , p = .776, d= 0.111; and no significant interaction between the CSD 
and delegation conditions was found, F(l, 12) = .316,/; =.584. For mean losses of separation and 
standard error per condition, see Table 1. 


Table 1. Loss of Separation Means and Standard Errors 


Condition 

Losses of Separation 

Mean 

S.E 

Basic CSD, Extended Delegation 

0.462 

0.215 

Basic CSD, Full Delegation 

0.308 

0.133 

Conflict Detection CSD, Extended 
Delegation 

0.462 

0.183 

Conflict Detection CSD, Full Delegation 

0.538 

0.268 


3.2 UAS Operator Workload Probes 

UAS operator workload probes were analyzed by their ratings and by the response times of the 
pilots. The workload ratings were collected on a scale of one (low) to seven (high). When analyzing 
probe ratings, if the pilot did not respond to the probe within the allotted time span of three minutes, 
the blank scores were adjusted to a score of seven (high). When analyzing probe response times, 
time outs (participant did not answer probe before the next probe was presented) were not included 
in the analysis. 

3.2. 1 Probe Ratings 

Results showed no significant differences in the ratings between the basic and conflict detection 
present CSD condition F( 1, 12) = 3.390 , p = .090, d= -1.107, though ratings did trend towards 
significance with conflict detection present condition (M= 4.012, SE = .247) having higher 
workload ratings than the basic condition (M= 3.527, SE = .218). No significant differences were 
found between the extended and full levels of delegation, F(l,12) = 1 .880, p = .195, d= 0.632; and 
no significant interaction between CSD and delegation conditions was found, F(l, 12) = .282, p = 
.605. Across all conditions, the overall workload probe scores were just below the average rating of 
four (M= 3.77, SD = 1.65). 
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3.2.2 Response Times 

Results showed no significant differences in the response times to workload probes between the 
basic and conflict detection present CSD condition F(l, 12) = .015,/? = .906, d= -2.105; no 
significant differences were found between the extended and full levels of delegation, F(l,12) = 
.195, p = .667 , d= 7.314; and no significant interaction between CSD and delegation conditions was 
found, F(l, 12) = 1.305,/) = .276. 

3.3 UAS Operator NASA TLX 

The NASA TLX is a self- assessed workload measure that includes ratings on six dimensions; 
mental, physical, temporal, performance, effort and frustration (Hart, & Staveland, 1988). The 
overall and individual subscales of the NASA TLX were analyzed. 

3.3. 1 Overall Workload 

Results showed no significant differences in the overall workload ratings between the basic and 
conflict detection present CSD conditions F(l, 12) = .003,/) = .960, d= 0.162. No significant 
differences were found between the extended and full levels of delegation, F(l,12) = 3.408 , p = 
.090, d= 2.748, though results trended towards significance with the extended delegation condition 
(M= 3.122, SE = .228) having higher workload ratings than the full delegation condition (M = 
2.782, SE = .211). No significant interaction was found between CSD and delegation conditions, 
F(l, 12) = .041, p =.842. While no significant differences were found for overall workload, the 
scores were on average low; M= 2.95, SE = .199 (Figure 14). 


Overall UAS Operator Workload 



1 1 1 1 

Basic & Basic & Full Conflict Conflict 
Extended Detection & Detection & 
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CSD and Delegation Condition 


Figure 14. UAS operator overall workload NASA TLX. 
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3.3.2 Individual Subscale 


The NASA TLX is separated into six individual subscales: mental, physical, temporal, performance, 
effort and frustration. The level of delegation had a significant effect on temporal workload, F(l, 12) 
= 10.958,/) = .006, d- 1.634, with full delegation ( M= 2.962, SE = .302) having significantly lower 
workload scores than extended delegation (M= 3.808, SE = .292). See Figure 15. 


Mean UAS Operator Temporal Workload 
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Figure 15. Mean UAS operator temporal workload. 


The effect of the level of delegation approached significance for frustration, F(l, 12) = 3.770,/) = 
.076, d= 1.125, with full delegation (M= 2.038, SE = .302) having lower workload scores than 
extended delegation (M= 2.696, SE = .440). All other subscale results were not significant (see 
Table 2). 


Table 2. UAS Operator NASA TLX Significance Summary Table 



NASA TLX Subscale 


Mental 

Physical 

Temporal 

Performance 

Degradation 

Effort 

Frustration 

CSD 

p = .947 

p = .488 

p= .719 

p= .715 

p = .802 

p = .706 

Delegation 

p = .157 

p= .137 

p = .006 

p = .730 

p = .279 

p = .076 

CSD* 

delegation 

p = .895 

p = .273 

p = .656 

p = .750 

p = .899 

p= .917 


While the ratings for mental, physical, effort and performance degradation were not found to be 
significant, the workload scores were found to be on average low to normal. Mental (M = 4.13, SE = 
.256), physical (M = 1 .36, SE = .299), effort (M = 4.09, SE = .269) and performance degradation (M 
= 2.36, SE = .353) respectively. 
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3.4 UAS Operator Situation Awareness Probes 

UAS operator situation awareness probes were analyzed for response accuracy and response time 
using 2x2 within subjects design ANOVAs. When analyzing probe response times, time outs 
(participant did not answer a probe question before the next probe was presented) were removed 
from analysis. 

3.4. 1 Probe Accuracy 

There were six probes presented to the UAS operator in each trial. Results were coded by accuracy 
with a score of one given to accurate responses, and a score of zero given to incorrect responses. The 
total amount of accurate responses per trial was then used in analysis. 

Results showed highly significant differences in probe accuracy between the basic and conflict 
detection present CSD conditions F(l, 12) = 452.107 , p < .001, d= -11.62, with the conflict detection 
present condition (M = 4.423, SE = .178) having significantly higher accuracy ratings than the basic 
condition (M= .840, SE = .026). See Figure 16. 
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Figure 16. Situation awareness probe accuracy. 


No significant differences were found between the extended and full levels of delegation, F(1 ,1 2) = 
.082 , p = .779, d- -0.146; and no significant interaction between CSD and delegation conditions was 
found, F( 1, 12) = .033, p =.859. 

3.4.2 Probe Response Time 

Results showed no significant differences in the response times between the basic and conflict 
detection present CSD condition F(l, 12) = 3.85 1 , p = .073, d= -0.618, although response times did 
approach significance with the conflict detection present mode (M = 25.994 seconds, SE = 3.889) 
having quicker response times than the basic mode (M = 33.631 , SE = 5.144). No significant 
differences were found between the extended and full levels of delegation, F(l,12) = A61 ,p = .690, 
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d= 0.891; and no significant interaction between CSD and delegation conditions was found, F( \ , 12) 
= .136, p =.719. 

3.5 UAS Operator Situation Awareness Questionnaires 

As the experimental blocks were based on CSD condition, situation awareness questionnaires were 
given out after each block on a scale from 0 (low) to 7 (high). In each block (basic or conflict 
detection present), participants performed one trial each of extended and full delegation. Situation 
awareness scores were analyzed using a 2x2 repeated measures design examining the effect of the 
CSD condition and the order of CSD presentation (condition presented first or second). 

Results showed no significant differences were found between the CSD condition, F(l,5) = .649 , p = 
.457, d= 1.17; and no significant interaction between order presented and CSD conditions was 
found, F( 1, 5) = .361 , p =.574. Significant differences in pilot perceived situation awareness 
between the order of CSD presentation F(1 , 5) = 9.826, p = .026, d= -0.444 were found, with the 
first experimental block presented (M = 5, SE = .302) having significantly lower perceived situation 
awareness ratings than the second experimental block presented (M= 5.241 ,SE= .290). This is not 
a surprising result as the UAS operators had more practice with the simulator by the second 
experimental block, and had better knowledge on procedures associated with the use of the CSD and 
delegated separation. See Figure 17. 
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Figure 17. Post-block situation awareness ratings. 


3.6 Air Traffic Controller Workload Probes 

Air traffic controller workload probes were analyzed by their ratings and by the response times of 
the ATC per trial. The workload ratings were collected on a scale of one (low) to nine (high). All 
workload probes were responded to within their allotted time span, so no adjustments were made on 
the scores, and no response times were discarded during analysis. It is of note to point out that since 
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there were only two air traffic controllers in this experiment, each controlling for roughly half the 
UAS operator participants, the analyses have low statistical power. 

3.6. 1 Probe Ratings 

Results showed no significant differences in the ratings between the basic and conflict detection 
present CSD conditions F(l, 1) = .132 ,p = .778, d= 0.023. No significant differences were found 
between the extended and full levels of delegation, F(l,l) = 1 ,p= .50, d= 0.132. No significant 
interaction between the CSD and delegation conditions was found, F(l, 1) = 5.1 83, p = .263. While 
workload probe ratings were not found to be significantly different, the overall workload ratings 
were manageable (M= 5 .913, SE = .220) and on par with typical workload probe ratings from 
previous studies performed in the FDDRL with air traffic controller participants (Johnson el al., 
2012; Battiste et al. 2012). See Figure 18. 



Figure 18. ATC workload probe ratings. 


3.6.2 Probe Response Times 

Results showed no significant differences in the response times between the basic and conflict 
detection present CSD conditions F(l, 1) = 3.817 , p = .301, d= 0.212. Results showed no significant 
differences in the response times for the levels of delegation F(l, 1) = .037 , p = .879. No significant 
interaction between the CSD and delegation conditions was found, F(l, 1) = .011 , p =.933. 

3.7 Air Traffic Controller NASA TLX 

The NASA TLX is a self- assessed workload measure that includes ratings on six dimensions; 
mental, physical, temporal, performance, effort and frustration (Hart, & Staveland, 1988). The 
overall and individual subscales were analyzed. It is of note to point out that since there were only 
two air traffic controllers in this experiment, each controlling for roughly half the UAS operator 
participants, the analyses have low statistical power. 


28 


3. 7. 1 Overall Workload 


Results showed no significant differences in the overall workload ratings between the basic and 
conflict detection present CSD conditions F(l, 1) = 29.388 ,p = .116, d= -0.2. No significant 
differences were found between the extended and full levels of delegation, F(l,l) = 25.733 ,p = 
.124. No significant interaction between the CSD and delegation conditions was found, F(l, 1) = 
.801 ,p = .535. While NASA TLX workload ratings were not found to be significantly different, the 
overall workload ratings were manageable (M= 4.788, SE = .260). See Figure 19. 
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Figure 19. ATC overall workload NASA TLX. 


3.7.2 Individual Subscale 

The effect of CSD condition approached significance for temporal workload, F( \ , 1) = 64, p = .079, 
d= -0.195, with the basic condition (M = 4.982, SE = .232) having lower workload scores than the 
conflict detection present condition (M = 5.077, SE = .244). All other subscale results were found to 
be not significant (see Table 3). 


Table 3. ATC NASA TLX Significance Summary Table 



NASA TLX Subscale 


Mental 

Physical 

Temporal 

Performance 

Degradation 

Effort 

Frustration 

CSD 

p= .126 

p = .500 

p = .079 

p = .830 

p= .357 

p = .677 

Delegation 

p = .304 

p = .500 

p= .179 

p = .170 

p = .390 

p = .279 

CSD* 

delegation 

p = .667 

p = .500 

p= .578 

p = .371 

p = .637 

p = .616 


While the ratings for mental, physical, temporal, performance degradation, effort and frustration 
were not found to be significant, the workload scores were found to be on average normal to 
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manageable. Mental (M = 5.76, SE = .467), physical (M = 4.02, SE = 1 .02), temporal (M = 5.03, SE 
= .238), performance degradation (M = 4.15, SE = .179), effort (M = 5.27, SE = .63) and frustration 
(M = 4.49, SE = .05) respectively. 

3.8 Subjective Questions 

The UAS operators and pseudo air traffic controllers were given post-trial questions pertaining to 
their overall workload, acceptability of reroutes, acceptability of final flight path, and the workload 
associated with their interactions with each other (UAS operator to pseudo controller) on a scale 
from zero (low) to seven (high). Ratings were analyzed using 2 (CSD condition) x 2 (level of 
delegation) ANOVAs. 

Additional questions on UAS operator and pseudo-controller workload, situation awareness, and 
acceptability and preference ratings were given to the participants at the end of the simulation. The 
post-simulation subjective questions were given on a scale from 0 (easy or acceptable) to 7 (hard or 
unacceptable). As the post-simulation questionnaires were collected at the end of the day after all 
trials were completed, analyses could not be performed to determine the effect of CSD condition and 
level of delegation experienced in each trial. The UAS operator responses are provided with the 
percentage of participants who rated that answer provided. 

3.8. 1 UAS Operator Subjective Post-Trial Questions 

Level of delegation was seen to have a significant effect on the difficulty to interact with air traffic 
control, F(l, 12) = 6.789,/) = .023, d- 2.369, with full delegation (M= .577, SE = .195) having 
significantly lower difficulty scores than extended delegation (M = 1 .808, SE = .469). While there is 
a significant difference in the mean, it is of note that both conditions have relatively low difficulty 
ratings. See Figure 20. 
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Figure 20. UAS operator difficulty interacting with ATC. 
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All other UAS operator post-trial subjective question results were found to be not significant (see 
Table 4). 


Table 4. UAS Operator Subjective Questions Significance Summary Table 



UAS Operator Subjective Questions 


Difficulty to 
Interact with ATC 

Acceptability of 
Traffic Reroutes 

Acceptability of 
Final Flight Path 

Overall 

Workload 

CSD 

p = .201 

p = .797 

p= .517 

p= .814 

Delegation 

p = .023 

p = .235 

p = .337 

p = .119 

CSD* Delegation 

p = .487 

p= .919 

p = .645 

p = .897 


While the ratings for both the acceptability of the UAS operators’ final flight path, and the 
acceptability of their reroutes to avoid traffic conflicts were not found to be significant, this was due 
to overall ratings of acceptability; M = 2.59, SE = .30 and M = 2.63, SE = .28 respectively. 

3.8.2 UAS Operator Post-Simulation Subjective Questions 

UAS operator post-simulation ratings are discussed by question type: perceived safety, workload 
associated with UAS, CSD usefulness and ease of use, CSD workload and willingness to use, and 
the amount of information displayed on the CSD. The tables show the responses given by the UAS 
operators and the percentages of participants who gave that response. 

While the UAS operators were mixed on their opinions of flying in a shared environment with UAS 
in the current NAS, over 60% of the participants were willing to share the airspace with properly 
trained unmanned vehicle operators. The importance of training is further emphasized with all 
participants rating that they gained more confidence operating the UAS and working in delegated 
separation conditions as they gained more practice. See Table 5 for all safety ratings and response 
percentages (rounded to the tenth). 

The post-simulation workload ratings reflect those collected via probes and the NASA TLX with 
overall ratings found to be low to manageable. A trend in level of delegation can be seen in the post- 
simulation ratings that match with other workload measures; full delegation was found to create 
lower UAS operator workload than extended delegation. See Table 6 for all workload ratings and 
response percentages (rounded to the tenth). 

The willingness to use CSD ratings shows that the UAS operators are more willing to fly a UAS in 
the NAS when using the CSD, with about 76% of the pilots being unwilling to fly a UAS without 
one. UAS operators are also more willing to fly a UAS in the NAS when conflict detection is present 
on the CSD. See Table 7 for CSD use ratings and response percentages (rounded to the tenth). 
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Table 5. UAS Operator Post-Simulation Perceived Safety Ratings 


Subjective Question 

Response Percentages 

I believe flying in a shared environment 
with UAS would be acceptable in the 
current environment. 

1.1% Strongly Disagree 
23.1% Somewhat Disagree 
15.4% Neither Disagree nor Agree 
30.8% Somewhat Agree 
23.1% Strongly Agree 

With appropriate training for UAS pilots, I 
would be willing to fly in an airspace that 
included UAS. 

23.1% Somewhat Disagree 
15.4% Neither Disagree nor Agree 
53.9% Somewhat Agree 
7.7% Strongly Agree 

I gained comfort with my ability to respond 
to the reroute orders as I gained more 
practice . 

15.4% Somewhat Agree 
84.6% Strongly Agree 

I gained comfort with my ability to operate 
the UAS under the extended delegated 
separation as I gained more practice. 

69.2% Somewhat Agree 
30.8% Strongly Agree 

I gained comfort with my ability to operate 
the UAS under full delegated separation as I 
gained more practice . 

38.5% Somewhat Agree 
61.5% Strongly Agree 


Table 6. UAS Operator Post-Simulation Workload Ratings 


Subjective Question 

Response Percentages 

I believe the overall WL associated with the 
concept of UAS in the NAS is manageable. 

7.7% Strongly Disagree 
7.7% Somewhat Disagree 
23.1% Neither Disagree nor Agree 
30.8% Somewhat Agree 
30.8% Strongly Agree 

Flying a UAS in extended delegated 
separation would not be possible due to 
high WL. 

7.7% Strongly Disagree 
46.2% Somewhat Disagree 
1.1% Neither Disagree nor Agree 
23.1% Somewhat Agree 
15.4% Strongly Agree 

Flying a UAS in full delegated separation 
would not be possible due to high WL. 

30.8% Strongly Disagree 
15.4% Somewhat Disagree 
15.4% Neither Disagree nor Agree 
23.1% Somewhat Agree 
15.4% Strongly Agree 

I was unable to successfully perform my 
mission due to the high WL associated with 
the rerouting events. 

23.1% Strongly Disagree 
30.8% Somewhat Disagree 
46.2% Somewhat Agree 

I was unable to successfully perform my 
mission due to the high WL associated with 
avoiding conflicts. 

23.1% Strongly Disagree 
15.4% Somewhat Disagree 
30.8% Neither Disagree nor Agree 
23.1% Somewhat Agree 
7.7% Strongly Agree 
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Table 7 UAS Operator Willingness to Use CSD Ratings 


Subjective Question 

Response Percentages 

I would be willing to fly center airspace 
with a UAS without using the CSD. 

53.9% Strongly Disagree 
23.1% Somewhat Disagree 
15.4% Neither Disagree nor Agree 
7 .7% Somewhat Agree 

I would be willing to fly center airspace 
with a UAS using the basic mode of the 
CSD. 

15.4% Strongly Disagree 
30.8% Somewhat Disagree 
23.1% Neither Disagree nor Agree 
30.8% Somewhat 

I would be willing to fly center airspace 
with a UAS using the conflict detection 
present mode of the CSD. 

7.7% Strongly Disagree 
7.7% Somewhat Disagree 
23.1% Neither Disagree nor Agree 
23.1% Somewhat Agree 
30.8% Strongly Agree 
7.7% NA 

Use of the CSD will enhance the safety of 
flying UAS in the NAS. 

23.1% Moderate Enhancement 
46.2% Intermediate Enhancement 
30.8% Large Enhancement 


The CSD usability ratings show an overall favorable view towards the use of the CSD. Across all 
participants, the CSD was used to determine reroutes and the information displayed on the CSD in 
both the basic and conflict detection modes was found to be useful. Overall, more participants found 
the conflict detection mode to be very useful when compared to the basic mode. Over 50% of the 
participants found their workload to be lower in the conflict detection mode than in the basic mode. 
See Table 8 for all CSD usability ratings and response percentages (rounded to the tenth). 

Questions were also asked of the UAS operator participants on the information displayed via the 
CSD. Overall, participants found more necessary information displayed in the conflict detection 
present mode than in the basic mode. See Table 9 for CSD information displayed ratings and 
response percentages. 

Open-ended questions were also asked of the UAS operators for what type of information they 
would like to see that was not available in the scenarios presented. The most common responses 
were to have the full flight plan of the surrounding aircraft available for assistance in conflict 
detection and avoidance, and to have an overlay of instrument and approach airways. While not used 
in this experiment in order to create a simulation environment closer to the airspace systems used in 
the current environment, it is worth noting that both of these capabilities are available on the Ames 
3D CSD and have been found to greatly assist in separation assurance. 
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Table 8. UAS Operator CSD Usability Ratings 


Subjective Question 

Response Percentages 

Did you use the CSD when determining 
reroutes. 

100% Yes 

How useful was the display for extracting 
airspace information in basic mode. 

15.4% Neither Useless or Useful 
76.9% Somewhat Useful 
7.7% Very Useful 

How useful was the display for extracting 
airspace information in conflict detection 
mode. 

15.4% Somewhat Useless 
7.7% Neither Useless or Useful 
61.5% Somewhat Useful 
23.1% Very Useful 

How useful was the conflict detection mode 
compared to the basic mode in supporting 
interactions with ATC. 

15.4% Somewhat Useless 
38.5% Neither Useless or Useful 
46.2% Somewhat Easy 

How easy was the display to use in the basic 
mode giving the mission requirements. 

15.4% Somewhat Difficult 
38.5% Neither Difficult or Easy 
46.2% Somewhat Easy 

How easy was the display to use with 
conflict detection present given the mission 
requirements. 

7.7% Somewhat Difficult 
38.5% Neither Difficult or Easy 
30.8% Somewhat Easy 
23.1% Very Easy 

What was your WL when using the conflict 
detection present CSD mode vs. the basic 
mode. 

23.1% Somewhat More Difficult 
23.1% Neither Difficult or Easy 
30.8% Somewhat More Easy 
23.1% Much More Easy 


Table 9. UAS Operator CSD Information Displayed Ratings 


Subjective Question 

Response Percentages 

Was all of the necessary information 
available from the CSD in basic mode. 

15.4% Somewhat Less Available 
30.8% Neither Missing or Available 
46.2% Somewhat Available 
7.7% Completely Available 

Was all of the necessary information 
available from the CSD in the conflict 
detection present mode. 

15.4% Somewhat Less Available 
23.1% Neither Missing or Available 
30.8% Somewhat Available 
30.8% Completely Available 


3.8.3 Air Traffic Controller Post-Trial Subjective Questions 

It is of note to point out that since there were only two air traffic controllers in this experiment, each 
controlling for roughly half the UAS operator participants, the analyses have low statistical power. 

The level of delegation was seen to have a highly significant effect on how acceptable air traffic 
controllers found the UAS operators’ reroutes to be, F(l, 1) = 504 1 , p = .009, d= 7.723, with the 
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extended delegation condition’s reroutes (M= 1.42, SE = .006) being rated as less acceptable to air 
traffic controllers than the full delegation condition (M= 1 ,SE= .000). See Figure 21. 
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Figure 21 . ATC reroute acceptability ratings. 


Across all participants and 100% of the trials, both air traffic controllers preferred full delegation to 
extended delegation and current day operations. All other subjective question results were found to 
be not significant (see Table 10). 


Table 10. ATC Subjective Questions Significance Summary Table 


ATC Subjective Questions 


Difficulty to 
Interact with 
UAS Operator 

Acceptability of 
Traffic Reroutes 

Difficulty Meeting Flow 
and Separation 
Requirements with UAS 

Overall 

Workload 

CSC 

p = .634 

p = .732 

p= .105 

p = .242 

Delegation 

p = .105 

p = .009 

p = .630 

p = .264 

CSD* 

Delegation 

p = .830 

p = .883 

p = .559 

p = .500 


While the post-trial air traffic controllers’ subjective ratings were overall not significant, the ratings 
themselves are worth note. In the difficulty to interact with UAS operator, the scores were overall 
acceptable (M= 1.42, SE = .006). The ratings for difficulty in maintaining flow and separation with 
a UAS present in sector were low to manageable (M = 3.49, SE = 2.10). The overall workload scores 
were high, but manageable (M= 5.81, SE = .399). The higher overall workload scores when 
compared to the UAS operators’ ratings (M= 3.98, SE = .29) are not surprising as the controllers 
maintained positive control over their sector while working with the UAS in either extended or full 
delegation. 
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3.8.4 Air Traffic Controller Post-Simulation Subjective Questions 

Air traffic controller questionnaire responses are discussed by question type: sector management 
queries, the controllers’ perceived capabilities of the UAS operators, and the perceived airspace 
safety and level of delegation preference. The tables show the responses given by the air traffic 
controllers, and the percentages of trials (13, one per UAS operator) that were given the responses. 

Multiple questions were asked to investigate the air traffic controllers’ experiences performing 
sector management (Table 1 1). Workload levels involved with managing a sector with an unmanned 
aircraft were found to be on par with or higher than normal current day operations. Meeting flow and 
separation requirements when the UAS operator was in extended delegation was seen to be between 
somewhat difficult and very difficult, on average. Full delegation was seen to be easier for the 
controllers to maintain flow and separation, with over half of the responses being rated as not at all 
difficult. In both extended and full delegation, controllers believed the UAS operator was able to 
maintain separation 92.3% of the time. 

The responses for whether or not the UAS created problems for the controllers’ managing their 
sectors were split, with just over half of the responses saying the UAS did not create problems. 

When asked if the UAS required special handling, 100% of the responses said yes, with just under 
70% of the responses saying that special handling was used between 0 - 25% of the time. Open- 
ended questions were asked on strategies used for special handling, with the average response being 
to use altitude restrictions for aircraft descending into the UAS’s flight level. See Table 1 1 for air 
traffic controller sector management question ratings and response percentages (rounded to the 
tenth). 

The controllers’ overall perception of UAS operator capabilities were on par with their perception of 
manned aircraft pilot capabilities. All UAS operator participants were found to have enough 
knowledge of the airspace and procedures to communicate and respond to the air traffic controllers’ 
instructions. This is not surprising, as all UAS operator participants were required to have at 
minimum a private pilot certificate. See Table 12 for air traffic controller perception of UAS 
operators’ capabilities ratings and response percentages. 

Across all participants, the air traffic controllers preferred full delegation. The controllers rated the 
perceived level of safety in the air transportation system with a UAS present as having on average 
the same level of safety as compared to normal operations. Matching the workload ratings of the air 
traffic controllers, full delegation was perceived as being safer than extended delegation, with 92.3% 
of the trials being considered the same level of safety as normal operations in the current airspace. 
See Table 13 for air traffic controller perceived safety and preference ratings and response 
percentages (rounded to the tenth). 

The 100% preference across participants by the air traffic controllers for full delegation correlates 
with the 92.3% rating of UAS flying under full delegation having the same perceived level of safety 
compared to the controllers’ perceptions of normal current day operations, as well as the better 
controller acceptability ratings of UAS operator reroutes when in full delegation. 
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Table 11. ATC Sector Management Questions 


Subjective Question 

Response 

WL managing a sector with a UAS in it 
compared to normal operations with only 
manned aircraft. 

30.8% Neither Higher or Lower 
46.2% Somewhat Higher 
23.1% Much Higher 

Difficulty maintaining flow and separation 
with the UAS in extended delegated 
separation compared to normal operations. 

7.7% Not at all Difficult 
38.5% Somewhat Difficult 
53.9% Very Difficult 

Difficulty maintaining flow and separation 
with the UAS in full delegated separation 
compared to normal operations. 

53.9% Not at all Difficult 
46.2% Somewhat Difficult 

The UAS operator was able to maintain 
separation in extended delegation. 

92.3% Yes 
7.7% No 

The UAS operator was able to maintain 
separation in full delegation. 

92.3% Yes 
7/7% No 

Did the UAS aircraft performance create 
problems for managing your sector? 

46.2% Yes 
53.9% No 

Did the UAS require special handling? If yes, 
what percentage of the scenario time? 

100% Yes 

69.2% 0-25% of time 
30.8% 25-50% of time 


Table 12. ATC Perception of UAS Operators’ Capabilities 


Subjective Question 

Response 

How immediately did UAS operators 
respond to instructions compared to 
manned pilots. 

7.7% Much Less Immediately 
38.5% Somewhat Less Immediately 
53.9% Same As Manned 

How appropriately did UAS operators 
respond to instructions compared to 
manned pilots. 

7.7% Much Less Appropriate 
15.4% Somewhat Less Appropriate 
76.9% Same as Manned 

Did UAS operators use correct 
terminology when communication, 
compared to manned pilots. 

7.7% Much Less Use 
61.5% Somewhat Less use 
30.8% Same as Manned 

The UAS operators have enough 
knowledge of the airspace and procedures 
to communicate. 

100% Yes 

The UAS operators have enough 
knowledge of the airspace and procedures 
to respond to instructions. 

100% Yes 
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Table 13. ATC Perceived Safety and Preference Ratings 


Subjective Question 

Response 

Preferred level of delegation of separation. 

100% Full Delegation 

Perceived level of safety of air transportation 
system with UAS in extended delegated 
separation compared to normal operation. 

53.9% Same Level of Safety 
46.2% Somewhat Less Safe 

Perceived level of safety of air transportation 
system with UAS in full delegated separation 
compared to normal operations. 

92.3% Same Level of Safety 
7.7% Somewhat Less Safe 


4.0 DISCUSSION 

The purpose of this thesis was to determine the feasibility of UAS performing delegated separation 
in the NAS. The study was a 2 x 2 within- subjects design, with level of delegation (extended 
delegation vs. full delegation) and traffic display information level (basic— traffic only vs. conflict 
detection present) as independent manipulations. The effects of delegated separation and traffic 
display information level were collected through objective and subjective measures of performance, 
workload and situation awareness. 

4.1 Loss of Separation 

Loss of separation data was analyzed by looking at the closest horizontal and vertical distances 
intruding aircraft reached in regards to the UAS per trial. A loss of separation was determined to 
have occurred when the minimum distances reached between the UAS and the intruder were within 
a distance of 5 nm horizontal and 1 ,000 ft vertical. In this experiment, there were no significant 
effects on loss of separation; on average there was less than one loss of separation per trial (M = .44, 
SE=. 124). 

The low occurrence of losses of separation across the conditions implies that the addition of more 
information on the CSD, and the addition of more responsibility for separation assurance given to 
the UAS operator does not increase the chance of a loss of separation occurring. This finding is 
important for safety issues associated with UAS flying in the NAS as the potential increase of loss of 
separations associated with UAS flying in a manned sector is a main safety concern. While losses of 
separation did occur, this data should be interpreted cautiously as this study contained experimental 
flight paths specifically designed to create conflicts. Further research should be performed to gain a 
more in depth understanding of how a UAS impacts the overall safety, flow and management of 
flight operations in the NAS. 

4.2 UAS Operator Workload 

Workload was collected from the UAS operator in by using online workload probes and through use 
of the NASA TLX. Pilot workload probes were analyzed by their ratings and by the response times 
of the pilots. No significant differences were found between the levels of delegation or the levels of 
information shown on the CSD, though ratings did trend towards significance with the conflict 
detection present condition having higher workload ratings than the basic condition. No significant 
differences were found on response times. 
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This trend towards higher workload probe scores on the UAS operator in the conflict detection 
present mode is not surprising as the operators were presented with a higher amount of information 
that they were required to pay attention to, process, make a decision on whether or not to avoid the 
conflict presented, and perform an action based upon that decision. Overall, workload probe scores 
did not surpass the “average,” score of four. This result implies that even with the added workload of 
conflict detection and higher amounts of separation responsibility, pilots were able to perform their 
task without high workload pressure and with comparable response times across all conditions. 

Overall workload scores on the NASA TLX were also not significant, though the level of delegation 
was seen to have a significant effect on the temporal dimension of workload, and the effect of 
frustration approached significance. The data from all other subscales did not differ significantly 
across the four experimental conditions. In both the temporal and frustration dimensions of 
workload, full delegation had lower scores than extended delegation. These findings are supported 
by previous research done on delegated separation that shows a reduction in workload when pilots 
are given full separation responsibilities (Krozel & Mogford, 2001; Lee, et. al., 2003., Johnson & 
Battiste, 2000). 

4.3 UAS Operator Situation Awareness 

Situation awareness measures were collected from the UAS operator in by using online situation 
awareness probes analyzed by accuracy and response times, and through the use of subjective 
questionnaires given post-CSD condition block. 

Probe results showed highly significant differences in accuracy between the basic and conflict 
detection present CSD conditions, with the conflict detection present condition facilitating the UAS 
operator situation awareness with significantly higher accuracy ratings than the basic condition. The 
increase of UAS operator situation awareness with the introduction of the conflict detection mode of 
the CSD is important to note. The lack of an out the window view associated with unmanned 
vehicles has been listed as a safety concern due to the potential for a decreased UAS operator 
situation awareness. The presence of the conflict detection present mode not only increased UAS 
operator situation awareness without significantly increasing UAS operator workload, but was found 
to induce higher UAS operator created reroute acceptability ratings from the air traffic controllers. 
These results show a shared benefit created by the conflict detection present mode of the CSD for 
both the UAS operator and the air traffic controller. 

Subjective questionnaire data collected post experimental block shows significant differences in 
pilot perceived situation awareness between the orders of CSD presentation, with the first 
experimental block presented having significantly lower perceived situation awareness ratings than 
the second experimental block presented. This finding is not surprising as the UAS operators had 
already completed two full trials and multiple training scenarios before they began the second 
experimental block, and had become accustomed to the apparatus and procedures associated with the 
study. 

Probe results show an increase in situation awareness correlating to the increase of information 
being presented on the CSD, with no significant effects on response times or workload. Subjective 
questionnaire results do not show the same significant effect of CSD on ratings, though the data 
trend of higher conflict detection present ratings supports the probe findings. The finding that the 
second experimental block increases the feeling of subjective situation awareness regardless of 
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condition is not surprising as the operator has gained better knowledge of how the display works and 
what to look out for by way of conflicts. 

What is interesting is the trend of the conflict detection present CSD mode having a greater increase 
in situation awareness after being presented second, compared to the basic mode being presented 
second (Figure 22). 
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Figure 22. UAS operator perceived situation awareness. 


This increase in situation awareness indicates the potential for an effect of training. Pilots may 
become more adept at their task when initially trained on a basic system, and then trained on a more 
advanced system. This finding is supported by Billinghurst et. al (2011), who found a similar effect 
when training participants on basic systems before introducing them to more complex NextGen 
systems . 

These findings suggest that the addition of a conflict detection tool to the CSD has the potential to 
increase situation awareness without negatively affecting workload or pilot response times. 
Moreover, when training pilots on advanced displays concepts, the addition of learning a basic 
version of the display first may foster a more effective and efficient use of the advanced system. 
Further research is required to more accurately determine the effects of flight deck display training 
and content. 

4.4 Air Traffic Controller Workload Probes 

ATC workload probes were analyzed by their ratings and by the response times of the ATC per trial 
No significant differences were found in workload probe scores, though both ATC had manageable 
workload ratings throughout the scenarios. No significant differences were found in probe response 
times. 

The NASA TLX results showed no significant differences in the overall workload ratings between 
the basic and conflict detection present CSD condition, no significant differences between extended 
delegation and full delegation, and no significant interaction between CSD and delegation 
conditions. While no significant differences were found between the extended and full levels of 
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delegation, results showed higher workload ratings in the extended delegation condition than the full 
delegation condition. 

The effect of CSD condition approached significance for temporal workload, with scores being 
lower in the basic condition. No significant differences or interactions were found for mental, 
frustration, effort, physical or performance degradation. While the subscale results were not 
significant, this might have been due to overall manageable ratings that did not change significantly 
throughout the conditions and trended towards lower scores in full delegation. This air traffic 
controller workload trend correlates to the same finding in UAS operator workload; extended 
delegation created a higher overall workload for participants . It is important to note that for both the 
air traffic controller and the UAS operator, extended delegation created an increase on temporal 
workload, significantly for the UAS operator, and an overall trend for higher ratings in this 
delegation condition. These results imply an advantage of full delegation to extended delegation that 
is shared between the air traffic controller and UAS operator. 

4.5 Post-Trial Subjective Questions 

The level of delegation was seen to have a significant effect on the UAS operator’s perceived 
difficulty to interact with air traffic control, as well as to have a correlating trend on the air traffic 
controllers’ perceived difficulty to interact with the UAS operators. For both types of participants, 
extended delegation created higher difficulty scores than full delegation. This is in support of the 
findings that show an increase in workload on both the air traffic controllers and the UAS operators 
when extended delegation was in use. The correlated workload and difficulty ratings show a shared 
advantage between the controllers and the pilots when full delegation is in use. 

The level of delegation was also seen to significantly affect the level of acceptability air traffic 
controllers had towards the UAS operators’ reroutes. When full delegation was in use, the air traffic 
controllers found the UAS operators’ reroutes to be significantly more acceptable. Higher air traffic 
controller acceptability of flight path reroutes is an additional benefit to reduced workload created by 
the use of full delegation. 

While multiple subjective questions were not found to have significant differences, the results are of 
note. For the UAS operators, the ratings for both the acceptability of their final flight path, and the 
acceptability of their reroutes to avoid traffic conflicts were not found to be significant. This was due 
to overall ratings of acceptability on a scale of zero (acceptable) to seven (unacceptable); M = 2.59, 
SE = .30 and M = 2.63, SE = .28 respectively. These ratings show an overall feeling of acceptability 
in flight path modifications from both the air traffic controllers and the UAS operators. 

Interestingly, while the CSD condition did not have a significant effect on how acceptable the UAS 
operators perceived their final flight path to be, the CSD condition was found to have a trend on how 
difficult it was for the air traffic controllers to maintain flow and separation requirements with the 
UAS present. Air traffic controllers found the basic CSD condition to increase the difficulty in 
maintaining flow and separation requirements in their sector than when the conflict detection present 
CSD condition was in use by the UAS operator. This trend was not significant however, and overall 
difficulty ratings for maintaining flow and separation were low to manageable, M= 3.49, SE = 2.10. 

This finding is of note as the conflict detection present condition made it easier for air traffic 
controllers to maintain flow and separation requirements while increasing the UAS operator 
situation awareness, and without significantly increasing UAS operator workload. 


41 


4.6 Post-Simulation Subjective Questions 

While the UAS operators were mixed on their opinions of flying in a shared environment with UAS 
in the current NAS, over 60% of the participants were willing to share the airspace with properly 
trained unmanned vehicle operators. The importance of training is further emphasized with all 
participants rating that they gained more confidence operating the UAS and working in delegated 
separation conditions as they gained more practice. This finding correlates with the increase in UAS 
operator situation awareness found in the second experimental block of scenarios. 

The post- simulation workload ratings for both UAS operators and air traffic controllers were found 
to correlate with probe and NASA TLX ratings with overall scores found to be manageable. A trend 
in level of delegation can be seen in the post-simulation ratings, and correlate with other workload 
measures; full delegation was found to create lower workload than extended delegation. Air traffic 
controller workload levels involved with managing a sector with an unmanned aircraft were rated to 
be on par with or somewhat higher than normal current day operations. Meeting flow and separation 
requirements when the UAS operator was in extended delegation was seen to be between somewhat 
difficult and very difficult, on average. Full delegation was seen to be easier for the controllers to 
maintain flow and separation, with over half of the responses being rated as not at all difficult. The 
workload ratings show a benefit to the use of full delegation that is shared between UAS operators 
and air traffic controllers. 

The responses for whether or not the UAS created problems for the air traffic controllers managing 
their sectors were split, with just over half of the responses saying the UAS did not create any 
problems. When asked if the UAS required special handling, 100% of the responses said yes, with 
just fewer than 70% saying that special handling was used between 0 - 25% of the time. While 
special handling was used a minor amount of the time, it is important to note that air traffic 
controller workload and losses of separation were not found to be significantly affected by the 
conditions. Air traffic controllers mentioned that the special handling they performed, primarily 
altitude restrictions, was the same type of handling they would do for other special use aircraft, such 
as those used in police patrols, and was not out of the ordinary. 

The UAS operators’ willingness to use the CSD ratings shows that they are more willing to fly a 
UAS in the NAS when using the CSD, with about 76% of the pilots being unwilling to fly a UAS 
without one. UAS operators are also more willing to fly a UAS in the NAS when conflict detection 
is present on the CSD, and all participants felt that the addition of a CSD in a UAS ground control 
station would increase the safety of UAS flight in the NAS. These findings are not surprising as the 
conflict detection present mode was found to significantly increase the accuracy ratings for situation 
awareness probes. Situation awareness is a vital aspect in conflict detection and avoidance, arguably 
increasing in importance for UAS operators, as they do not have an out the window view and rely 
more heavily upon their traffic display. 

The UAS operator CSD usability ratings show an overall favorable view towards the use of the 
CSD. Across all operators, the CSD was used to determine reroutes, and the information displayed 
on the CSD in both the basic and conflict detection modes was found to be useful. Overall, UAS 
operators found the conflict detection mode to be more useful when compared to the basic mode, 
which correlates with the increase in situation awareness and an increased willingness to fly a UAS 
ratings, as well as an increase in the air traffic controllers’ acceptability ratings for the UAS 
operators’ reroutes. Additionally, over 50% of the UAS operators found their workload to be easier 
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in the conflict detection mode than in the basic mode, a finding that is correlated across workload 
measures collected. 

UAS operators felt that more necessary information was available on the CSD when in conflict 
detection mode. When asked what information they would like to see that was not available in the 
simulation, UAS operators most commonly responded with being able to see the entire flight path of 
the surrounding aircraft and an overlay of instrument and approach airways. While not used in this 
experiment, it is worth noting that both of these capabilities are available on the Ames 3D CSD and 
have been found to assist in separation assurance and increased situation awareness. Further research 
should be performed to investigate if the display of aircraft trajectories and overlays of airways on 
the UAS operators’ CSD has a positive effect on the air traffic controllers’ reroute acceptability and 
difficulty in maintaining flow and separation ratings. 

The air traffic controllers’ overall perception of UAS operator capabilities were on par with their 
perception of manned aircraft pilot capabilities. All UAS operator participants were found to have 
enough knowledge of the airspace and procedures to communicate and respond to the air traffic 
controllers’ instructions. As the UAS operator participants were required to have at minimum a 
private pilot certificate, this was not a surprising finding. Further research should be done to 
investigate the effects of different levels of flight experience, certifications and video game use on 
UAS operation to better define what the requirements will be for UAS operators in the NAS. 

Across all participants, the air traffic controllers preferred full delegation 100% of the time. The 
controllers rated the perceived level of safety in the air transportation system with a UAS present as 
having on average the same level of safety as compared to normal operations. Full delegation was 
perceived as being safer than extended delegation by the air traffic controllers, a finding that is 
correlated with both the controllers’ preference, and the workload ratings of both the controllers and 
the UAS operators. This again suggests that the use of full delegation creates benefits experienced 
by both the UAS operator and the air traffic controller. 

4.7 Comparison of the Current Unmanned Systems Results to Manned Flight 

An important objective measure collected in this experiment was the number of losses of separation 
that occurred per condition within 5 nm horizontal and 1000 ft vertical. While this study found no 
significant differences between conditions, the number of LOS are comparable to other studies. A 
study by Fern, Kenny, Shively & Johnson (2012) measured baseline compliance rates for UAS 
operating in the current airspace system with the same sector and traffic patterns of this experiment, 
though air traffic control was responsible for maintaining separation. Results showed comparable 
rates of overall LOS (M = 0.292, SD = 0.464) to those collected in this study (M = 0.442, SD = 
0.725). Vu et al. (2012) studied delegated separation for manned aircraft using the same separation 
standards. Results from Vu et al. (2012) interestingly showed a higher rate of overall LOS than this 
study (M = 0.625), also with no significant differences in the rate of LOS between the different 
levels of delegation. While a comparable rate of LOS did occur in these studies, these data points 
should be interpreted cautiously as all flight paths were specifically designed to create conflicts. 

Previous studies focusing on manned aircraft performing delegated separation showed a decrease in 
the number of message exchanges and instructions between ATC and pilots (Vu et al., 2012; 
Grimaud, Hoffman & Zeghal, 2001; Zeghal, Grimaud, Hoffman & Rognin, 2001). The same result 
was found in the current study; there were fewer interactions between the UAS operator and ATC 
under delegated separation. The ATC preference for full delegation in this study is also consistent 
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with previous findings of controllers rating delegated separation to be an effective and useful tool in 
maintaining flow and separation (Grimaud, Hoffman & Zeghal, 2001). 

The ATC ratings collected in this study further indicated that controllers had an easier time of 
meeting flow and separation requirements. They also reported that UAS operator created reroutes 
were significantly more acceptable under full delegation. These ratings are consistent with previous 
findings in manned flight that report increases in efficiency based on time, distance, fuel 
consumption, straightness of trajectories, and closest point of approach (Grimaud, Hoffman & 
Zeghal, 2001; Lee et al., 2003). Additionally, faster response times from flight crews in full 
delegation to off nominal scenarios have also been found; flight crews were less likely to be passive 
and wait for the air traffic controller to intervene in the event of a loss of separation when ATC is 
not responsible (Prinzel et al., 2011; Johnson, Battiste, Delzell, Holland, Belcher & Jordan, 1997; 
Vu et al., 2012). 

Full delegation was associated with lower UAS operator workload ratings in this experiment. In the 
temporal and frustration dimensions of the NASA TLX, and post-simulation likert scales, full 
delegation resulted in lower workload ratings than extended delegation. These findings are 
consistent with previous research done on delegated separation when manned pilots are given full 
separation responsibilities (Krozel & Mogford, 2001; Lee et al., 2003, Johnson & Battiste, 2000; 
Prinzel et al., 2011). While ATC workload ratings were not found to be significantly different, the 
overall workload ratings were manageable (M = 5.913, SE= .220). They were also similar to typical 
workload probe ratings from previous studies performed in the FDDRL using the same workload 
probe scale with air traffic controller participants (Johnson el al., 2012; Battiste et al., 2012; Vu et 
al., 2012). 

When using the cockpit situation display, UAS operators reported that the conflict detection mode 
was more useful and comprehensive than the basic mode. This correlated with an increase in 
operator SA, higher ATC reroute acceptability ratings, and lower UAS operator post-sim WL 
ratings. These findings are consistent with previous findings of manned pilots having a high 
confidence in using cockpit displays of traffic information (CDTIs) for delegated separation 
(Domino, Tuomey, Mundra & Smith, 2010; Johnson, Battiste, Delzell, Holland, Belcher & Jordan, 
1997). 

4.8 Future Research Considerations 

Future research should be done to determine UAS operator requirements and training requirements, 
in addition to determining the optimal way to present conflict detection alerts. In this experiment, 
alerts were given to the UAS operator as far as 20 minutes out, which caused many UAS operator 
participants to voice frustrations at such early alerts. It is unknown what the proper timing for alerts 
should be for a UAS to successfully avoid a conflict as their flight characteristics (such as speed and 
maneuverability) are different than manned aircraft. Further research should be done, perhaps 
through the use of parametric studies, to investigate the effects of differing flight characteristics on 
alert usage and UAS operator responses. 

A more in-depth look at the conflict data in this experiment may help to shed light on this by an 
additional analysis determining how far out the intruders were before the alerts were given. It would 
also be useful to examine how long it took the UAS operator to respond to the alert, maneuver and 
avoid the conflict. Finally, it would be important to determine the threshold for distance/time that 
allowed the UAS operator to successfully avoiding the conflict and how many conflicts were 
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actively vs. passively avoided (active move or passive change in leg of flight path). These data sets 
could provide useful information for future studies to focus on determining what the proper time and 
distance thresholds are for providing UAS operators with alert notifications. The use of trajectory 
based vs. ballistic based conflict information should also be considered. Trajectory based conflict 
information should help to reduce the amount of false alarms, or unnecessary alerts, such as in the 
cases of passive conflict avoidance. 

5.0 Conclusion 

The purpose of this thesis was to determine the feasibility of UAS performing delegated separation 
in the NAS. The results of this study support the feasibility of UAS performing delegated separation 
in the NAS while providing areas in need of further research. Overall, adjusting the level of 
separation responsibility and amount of information available to the UAS operators on the CSD was 
not found to have an adverse effect on performance as shown by the low amounts of losses of 
separation. The results of the study show benefits related to the use of a more advanced CSD with 
conflict detection capabilities, and to the use of full delegation. 

In the more advanced conflict detection present mode of the CSD, pilots had significantly higher 
situation awareness without a significant negative effect on their reaction times or workload levels. 
While there was a trend toward higher workload ratings associated with the conflict detection 
present mode, it is possible that this slight increase could be ameliorated by training the pilot in a 
basic traffic display first. Post simulation workload ratings showed that over 50% of the UAS 
operators found their workload to be lower in the conflict detection mode as well. In addition to the 
increased UAS operator situation awareness and low workload ratings, air traffic controllers were 
more likely to rate the reroutes performed by the UAS operator as being acceptable. Air traffic 
controllers also reported having an easier time performing flow and separation requirements in their 
sector when the conflict detection present mode of the CSD was in use. 

In the full level of delegation, both UAS operators and air traffic controllers benefited from the 
transition of separation responsibility. A decrease in temporal workload, and a trend towards 
reduced frustration was found for both air traffic controllers and UAS operators in full delegation. 
Subjectively, both air traffic controllers and UAS operators reported that the interaction between 
them was easier in full delegation, and air traffic controllers preferred the UAS operator to have full 
delegation responsibilities 100% of the time across all of the trials. 

The use of a more advanced CSD and full delegation responsibilities given to the UAS operator 
were found to create significantly reduced workload, significantly increased situation awareness and 
significantly easier communications between the UAS operator and air traffic controller without 
significantly increasing the amount of losses of separation. 
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